Evaluation of Agent-Oriented Methodologies

The need for methodologies in the domain of agent-based system has been discussed by several studies [2, 3, 13, 14, 15]. For example, according to [13] AOSE is a key factor for introducing agent-based systems to the industry as an engineering approach. During the last decade, there was a blooming of agent-oriented methodologies. More than two dozen methodologies (see Appendix) were developed based on various theoretical grounds, such as, object-orientation and knowledge representation. Yet, only in the last few year the evaluation of these methodologies gains the research community attention, deriving by standardization efforts. Examining the various studies done on evaluation of agent-oriented methods, we found that there they are lacking in a systematic way of evaluating the methodologies. For example, a comparison among agent-oriented methodologies appears in [2]. That work examines the similarity between the models of the GAIA [23] and MAS-CommonKADS [12] methodologies. Yet, it does not explicitly evaluate these methodologies or provide techniques for doing so. A similar comparison is presented in [8], in which the authors compare between GAIA and MaSE [10] and concludes that MaSE is much more detailed than GAIA. However, they do not address drawbacks of MaSE nor do they provide guidelines for making a comparison between methodologies. The studies presented in [2] and [8] focus on the expressiveness (in terms of agent systems) of the methodologies they examined. However, those studies do not examine the software engineering criteria of the studied methodologies in the comparison they perform. Another effort for setting a basis for evaluating agent-oriented methodologies is presented in [7]. That work checks the level to which some specific qualitative features are present within a methodology using decision trees, it computes the level to which the evaluated method meets the needs of its users. It does so by allocating a weight to each feature of the method, grading each feature, and calculating a weighted average grade. However, the features according to which a methodology should be evaluated are not determined. In [24], the authors suggest an exemplar case study according to which the various methodologies could be evaluated. In addition to the case study, they list a set of questions to be asked about an agent-oriented methodology. However, the questions are somewhat vague and answering these questions may not lead to an understanding of what the right methodology is for a specific project, i.e., there is no framework for evaluating agent-oriented …